18 research outputs found
Mutation testing from probabilistic finite state machines
Mutation testing traditionally involves mutating a program in order to produce a set of mutants and using these mutants in order to either estimate the effectiveness of a test suite or to drive test generation. Recently, however, this approach has been applied to specifications such as those written as finite state machines. This paper extends mutation testing to finite state machine models in which transitions have associated probabilities. The paper describes several ways of mutating a probabilistic finite state machine (PFSM) and shows how test sequences that distinguish between a PFSM and its mutants can be generated. Testing then involves applying each test sequence multiple times, observing the resultant output sequences and using results from statistical sampling theory in order to compare the observed frequency of each output sequence with that expected
Recommended from our members
Testing from a stochastic timed system with a fault model
In this paper we present a method for testing a system against a non-deterministic stochastic finite state machine. As usual, we assume that the functional behaviour of the system under test
(SUT) is deterministic but we allow the timing to be non-deterministic. We extend the state counting method of deriving tests, adapting it to the presence of temporal requirements represented by means of random variables. The notion of conformance is introduced using an implementation relation considering temporal aspects and the limitations imposed by a black-box framework. We propose an algorithm for generating a test suite that determines the conformance of a deterministic SUT with respect to a non-deterministic specification. We show how previous work on testing from stochastic systems can be encoded into the framework presented in this paper as an instantiation of our parameterized implementation relation. In this setting, we use a notion of conformance up to a given confidence level
Extending stream X-machines to specify and test systems with timeouts
Stream X-machines are a kind of extended finite state machine used to specify real systems where communication between the components is modeled by using a shared memory.In this paper we introduce an extension of the Stream X-machines formalism in order to specify delays/timeouts.The time spent by a system waiting for the environment to react has the capability of affecting the set of available outputs of the system. So, a relation focusing on functional aspects must explicitly take into account the possible timeouts.We also propose a formal testing methodology allowing to systematically test a system with respect to a specification. Finally, we introduce a test derivation algorithm. Given a specification, the derived test suite is sound and complete, that is, a system under test successfully passes the test suite if and only if this system conforms to the specification
Using genetic algorithms to generate test sequences for complex timed systems
The generation of test data for state based specifications is a computationally expensive process. This problem is magnified if we consider that time con- straints have to be taken into account to govern the transitions of the studied system. The main goal of this paper is to introduce a complete methodology, sup- ported by tools, that addresses this issue by represent- ing the test data generation problem as an optimisa- tion problem. We use heuristics to generate test cases. In order to assess the suitability of our approach we consider two different case studies: a communication protocol and the scientific application BIPS3D. We give details concerning how the test case generation problem can be presented as a search problem and automated. Genetic algorithms (GAs) and random search are used to generate test data and evaluate the approach. GAs outperform random search and seem to scale well as the problem size increases. It is worth to mention that we use a very simple fitness function that can be eas- ily adapted to be used with other evolutionary search techniques
Passive Testing with Asynchronous Communications and Timestamps
We develop a formal passive testing framework for software systems
where parties communicate asynchronously. Monitors, placed in between the en-
tities, check that a certain property holds over the observations of the interaction
between users and the System Under Test (SUT). Due to the asynchronous nature
of communications, the trace observed by the monitor might differ from the one
produced by the SUT: the monitor observes inputs before they are received by
the SUT and outputs are observed after they are sent by the SUT. It is neces-
sary to take this into account in passive testing; otherwise we might obtain false
positives or false negatives. In order to better assess the real causality between
actions, we consider the case where each action is labelled with a timestamp giv-
ing the time when it was observed at the monitor. We also assume that we know
bounds on network latency and so the timestamps allow us to determine additional
causalities between actions. Our monitors are implemented as automata that take
into account communications being asynchronous. Our solution checks properties
against traces in polynomial time and has low storage requirements. Therefore,
our proposal is suitable for real-time passive testing.Research partially supported by the projects DArDOS (TIN2015-65845-C3-1-R
(MINECO/FEDER)) and SICOMORo-CM (S2013/ICE-3006)
Recommended from our members
An extended framework for passive asynchronous testing
In passive testing a monitor observes the trace (sequence of inputs and outputs) of the system under test (SUT) and checks that this trace satisfies a given property , potentially triggering a response if an incorrect behaviour is observed. Recent work has explored a variant of passive testing, in which we have a required property of the traces of the SUT and there is a first-in-first-out (FIFO) network between the SUT and the monitor. The problem here is that the trace observed by the monitor need not be that produced by the SUT. Previous work has shown how such asynchronous passive testing can be performed if the property is defined by a pair that represents the requirement that if trace is produced by the SUT then the next output must be from the set . This paper generalises the previous work to the case where the property is defined by a finite automaton
A tool supported methodology to passively test asynchronous systems with multiple users
Spanish MINECO/FEDER (grant number TIN2015-65845-C3-1-R); Region of Madrid (grant number S2013/ICE-3006)
Integration testing of heterotic systems
Computational theory and practice generally focus on single-paradigm systems, but relatively little is known about how best to combine components based on radically different approaches (e.g. silicon chips and wetware) into a single coherent system. In particular, while testing strategies for single-technology artefacts are generally well developed, it is unclear at present how to perform integration testing on heterotic systems: can we develop a test-set generation strategy for checking whether specified behaviours emerge (and unwanted behaviours do not) when components based on radically different technologies are combined within a single system? In this paper, we describe an approach to modelling multi-technology heterotic systems using a general-purpose formal specification strategy based on Eilenberg's X-machine model of computation. We show how this approach can be used to represent disparate technologies within a single framework, and propose a strategy for using these formal models for automatic heterotic test-set generation. We illustrate our approach by showing how to derive a test set for a heterotic system combining an X-machine-based device with a cell-based P system (membrane system)
Intraperitoneal drain placement and outcomes after elective colorectal surgery: international matched, prospective, cohort study
Despite current guidelines, intraperitoneal drain placement after elective colorectal surgery remains widespread. Drains were not associated with earlier detection of intraperitoneal collections, but were associated with prolonged hospital stay and increased risk of surgical-site infections.Background Many surgeons routinely place intraperitoneal drains after elective colorectal surgery. However, enhanced recovery after surgery guidelines recommend against their routine use owing to a lack of clear clinical benefit. This study aimed to describe international variation in intraperitoneal drain placement and the safety of this practice. Methods COMPASS (COMPlicAted intra-abdominal collectionS after colorectal Surgery) was a prospective, international, cohort study which enrolled consecutive adults undergoing elective colorectal surgery (February to March 2020). The primary outcome was the rate of intraperitoneal drain placement. Secondary outcomes included: rate and time to diagnosis of postoperative intraperitoneal collections; rate of surgical site infections (SSIs); time to discharge; and 30-day major postoperative complications (Clavien-Dindo grade at least III). After propensity score matching, multivariable logistic regression and Cox proportional hazards regression were used to estimate the independent association of the secondary outcomes with drain placement. Results Overall, 1805 patients from 22 countries were included (798 women, 44.2 per cent; median age 67.0 years). The drain insertion rate was 51.9 per cent (937 patients). After matching, drains were not associated with reduced rates (odds ratio (OR) 1.33, 95 per cent c.i. 0.79 to 2.23; P = 0.287) or earlier detection (hazard ratio (HR) 0.87, 0.33 to 2.31; P = 0.780) of collections. Although not associated with worse major postoperative complications (OR 1.09, 0.68 to 1.75; P = 0.709), drains were associated with delayed hospital discharge (HR 0.58, 0.52 to 0.66; P < 0.001) and an increased risk of SSIs (OR 2.47, 1.50 to 4.05; P < 0.001). Conclusion Intraperitoneal drain placement after elective colorectal surgery is not associated with earlier detection of postoperative collections, but prolongs hospital stay and increases SSI risk
Recommended from our members
Bounded reordering in the distributed test architecture
In the distributed test architecture, the system under
test interacts with its environment at multiple physically
distributed ports and the local testers at these ports do not
synchronise their actions. This presents many challenges and,
in particular, apparently incorrect behaviours can be the consequence
of an erroneous assumption about the exact order in
which actions were performed at different ports. In previous
work, we defined a conformance relation for the distributed
test architecture. Essentially, the system under test is faulty if
we observe a trace such that no admissible reordering of
the actions in could have been produced by the specification.
However, this notion can be weak if the compared traces might be
too different. This paper introduces conformance relations where,
for a given metric, a reordering is only considered if the distance
between the two traces is at most a certain bound k. We introduce
two different metrics and provide algorithms to construct finite
automata accepting these close, with respect to each metric,
sequences. We also study the computational complexity of the
two main problems associated with the new framework: deciding
whether a trace is accepted by the new automaton and deciding
whether one system conforms to a specification with respect to
the new conformance relation